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Abstract. One of the steps in software development is to test the software product. With the development of 
technology, the testing process has improved to automated testing, which reduces the impact of the human factor on 
error and speeds up testing. The main software products for testing are considered to be web applications, web services, 
mobile applications and performance testing. According to the testing pyramid, when testing web services, you need to 
develop more test cases than when testing a web application. Because automation involves writing software code for 
testing, the use of ready-made tools will speed up the software development process. One of the most important test 
indicators is the coverage of search functionality. The search functionality of a web application or web service requires 
a large number of cases, as you need to provide many conditions for its operation through the free entry of any 
information on the web page. 

There is an approach to data-based testing, which involves working with a test data set through files such as 
CSV, XLS, JSON, XML and others. However, finding input for testing takes a lot of time when creating test cases and 
automated test scenarios. It is proposed to use artificial data set generators based on real values and popular queries on 
the website to form a test data set. In addition, it is possible to take into account the probable techniques of developing 
test cases. 

It is proposed to conditionally divide the software for testing into several layers: client, test, work with data, 
checks and reports. The Java programming language has a number of libraries for working at each of these levels. It is 
proposed to use Rest Assured as a Restful client, TestNG as a library for writing tests with checks, and Allure report for 
generating reports. 

It is noted that the proposed approach uses artificial intelligence for automated selection of test cases when 
creating a test to diversify test approaches and simulate human input and behavior to maximize the use of cases. 


Keywords: automation testing, artificial intelligence, synthetic data, data generator, artificial datasets, web API 
testing, data-driven testing. 
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Anotauisa. OHM 13 eTaliB Ip pospoOui MporpamuHoro 3a0e3Me4eHHA € TECTYBAHHA MpOrpaMHOro MpOyKtTy. 
3 pO3BHTKOM TeXHOJOTi Mpolec TeCTyBaHHA YHOCKOHAIMBCA JO ABTOMATH30BaHOTO TeCTYBaHHA, AK 3MCHILYE BIVIMB 
JHOCbKOrO akTopy Ha HOMMWIKy Ta MpHuwBMAye TecTyBaHHa. OcHOBHHMH UporpaMHHMU MposxyKTaMu fA 
TECTYBAHHA BBaKAaIOTbCA BeEOSACTOCYHKH, BeOCepBICH, MOOWIbHI JOAATKM Ta TecTyBaHHA ecpeKTUBHOCTI. BiqnoBiqHO oO 
TipaMiqu TeCTyBaHHA, Mp TecTyBaHHi BeOcepBiciB MOTpIOHO po3poONATH OibuWIe TeCTOBUX BHMaAKiB, HK IA 
TeCTyBaHHA BeOsacTocyHkKy. OcKiJIbKH aBTOMAaTH3allid epeqOayae HallHcaHHa WporpaMHoro KOLy [id TecTyBaHHA, TO 
BHKOPHCTaHHA TOTOBHX iHCTPYMCHTIB UPHWBUAWMTb mpolec po3spoOKH mporpaMHoro 3a0esneueHHA. OHM 3 
HalBaxKIMBILIMX TeCTOBHX MWOKa3HHKIB BBaxKa€TbCA MOKPHTTA NMoWykKoBoro PyHKuionany. MouryKoput yHKuioHal 
BeO3acTOCYHKy 4 BeOcepBicy NoTpeOye oMmpalloBaHHA BeIMKOrO OOCATY BUMATKIB, OCKIIbKM MOTPIOHO MepeqOayaTu 
MHOXKHHY YMOB Horo PYHKUIOHYBaHHA Yepe3 BIIbHe BBeeHHA Oy {b-AKOI iH(popMaliil Ha BeOcTOpIHKy. 

IcHye Mixiq WO TecTyBaHHA Ha OCHOBI WaHux, AKU MepeqOauae poOoTy 3 TecTOBuM HaOopoM AaHUx Yepe3 
dbatinm tuny CSV, XLS, JSON, XML Ta inuti. IIpote, nouryk BxiqHUX aHUx [Ia TecTyBaHHA 3a0upae OaraTo yacy 
IIpH CTBOpeHHi TeCTOBHX BUHMaKiB Ta aBTOMAaTH30BaHHX TeCTOBHX ClleHapiis. 3anponoHoBaHO BHKOpHCTaTH 
reHepaTopu WTyyHoro Hadopy WaHHX Ha OCHOBi peayIbHHX 3Ha4eHb Ta MOMyIAPHUX 3alWTIB Ha BeOcaliTi DIA 
cbopMyBaHHa TecToBoro Habopy AaHux. KpimM Uboro, € MOXKIMBICTh BpaxyBaTH MMOBIPHI TeXHIKM PO3poOKH TeCTOBHX 
BHUAaIKIB. 

IIporpamue 3a0e3re4ueHHA [Id TeCTyBaHHA 3aMpOMOHOBaHO YMOBHO MOJiIMTH Ha JeKWIbKa pOwlapKiB: 
KJ€HTCbKHH, TECTOBHH, poOoTa 3 JaHHMU, MHepeBipKH Ta 3BITH. Y MOBi MporpamMyBaHHaA Javae pay OiOMoTeK AWA 
poOoTH Ha KOXKHOMYy 3 UWMX PpiBHiB. 3alpomoHoBaHO BuKopuctaTH REST Assured, 1k RESTful Client, TestNG aK 
OiOMIOTeKH JIA HalMcaHHA TeCTIB 3 MepeBipKaMH, a Takox Allure report 41d reHepyBaHHA 3BiTIB. 

Bigz3Ha4eHo, WO 3alIponoHOBaHH 3 ypaxyBaHHAM IWTY4HOro IHTeIeKTY MAXI Ait AaBTOMAaTH30BaHOrO MiAOopy 
TeCTOBHX BUMa{KiB MIpH CTBOPeHHI TeCTy ypi3HOMAHITHHTb TeCTOBI JjaHi, MaKCHMaJIbHO CTHMYJIIOBaTHMe BBe]eHHA 
Ta@HMX JIOMHO!LO Ta NOBCAIHKY [Id MOKPUTTA OIbIIOCTI BHMAKIB. 


Kso4oBi c10Ba: aBTOMaTH30BaHe TeCTYBaHHA, WITYYHHM 1HTeJIeKT, CHHTeTHYHI ai, reHepaTop aHUx, 
LITYWHH Hadip WaHux, TeCTyBaHHA BeOcepBicy, TeCTyBaHHA Ha OCHOBI aHHXx. 


Introduction popular in recent years. When generating 
Artificial intelligence involves checks for web services, you should take into 
modeling the ability of human intelligence by account the status of codes and responses in 
intelligent machines of various kinds. Today, JavaScript Object Notation (JSON) format. 
artificial intelligence is increasingly being Therefore, API testing involves understanding 
integrated into software quality assurance. and working on the Internet [6]. 
Such an approach is best suited for automated 
type testing. In addition, it is worth Literature review 
mentioning that the web service requires the In recent years, the use of artificial 
most attention when testing because intelligence in testing has been particularly 
malfunctions in the various Application intensively considered [1, 4]. There are a 
Programming Interfaces (APIs) can cause large number of materials for data generation 
errors in the work and and affect the status of using genetic algorithms with metaheuristic 
the web pages. The web service is used to search methods [11], and simple solutions, 
communicate with clients and the web server namely synthetic ones are described in [3, 8]. 
using the Internet [6]. It is also worth noting Moreover, different layers use machine 
that it is still possible to test a web page learning approaches, depending on _ the 
manually, and testing a web service with a implementation. The basic architecture of the 
large number of requests can take a lot of time framework extension consists of testing based 
and resources. Therefore, API testing requires on the data approach presented in [7]. 
writing test scripts. Regarding the process of data generation, the 
One of the main reasons for failing tests article [9] describes various approaches, such 
is frequent data changes. In test scenarios, you as random, target-oriented, intelligent, path 
need to specify the expected result to check oriented data generators. However, it is worth 
for convergence procedures. Usually, these considering the use of existing ready-made 
are usually specific values that can vary. libraries and capabilities in selecting the right 
Among the most common types of web data for the chosen data-driven testing 
services are REST, SOAP, XML-RPC. approach. 


In recent trends, the implementation of 
the REST - Restful architecture is the most 
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Data generation 


tayer 
Client layer <« 


Validation 
layer 


Fig.1. Basic layers of the framework 
for automated testing 


Figure | shows the main components 
that make up the framework for automated 
testing. The main types of API [2] and 
graphical user interface (GUI) [5] testing 
using different techniques for writing code are 
considered in the works. 

Testing can be applied to a pre-designed 
system for detecting defects [10]. This system 
has the ability to fully and partially search for 
data. 


Methods and Materials 

When developing a framework for 
automated testing, it is necessary to use 
several important libraries that simplify code 
writing. 

RestAssured library is designed to 
communicate with the web service. 

TestNG is considered to be the most 
well-known tool for setting up and writing 
test scripts due to a large number of 
possibilities for setting up the test. 

Allure report is the most convenient for 
reporting after testing. This reporting form 
displays trends, path rate, number of passed 
and failed tests. The obtained parameters help 
to determine the test indicators and present 
the results in a modern presentation. 

Among the main approaches to testing 
are behavior-driven, data-driven, test-driven, 
keyword-driven testing, and others. Since 
working with data is most used in data-driven 
testing, it is worth considering a method using 
the closest possible generated data set. 

A maven is a _ popular tool for 
assembling a framework with different phases 
of construction and automatic loading of 
external libraries. 

Assert J provides a set of advanced 
validations to flexibly process existing and 
expected results. 

The Apache POI library belongs to the 
Java API and is used to work with Microsoft 
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files. In the case of automated testing, the 
library is applied to read the generated values 
into tests. 

A synthetic method uses for filling an 
artificial data set of different resources. 


Synthetically | generated datasets are 
increasingly used for machine learning. The 
Java Faker library in the developed 


framework is used for the generation of this 
type of data. 

Test case design techniques are used 
from testing theory [13]. 


Problem Formulation 

Lack of data in testing is a fairly 
common problem faced by most researchers 
and software testers. This is especially 
common with search queries for picking up 
non-existent values and is time-consuming. 

In addition, to obtain information, you 
need to access the web service using Uniform 
Resource Locator (URL). To form a specific 
request, you need to build a path to the 
resource in the network. The main 
components of a URL are host, port, and set 
of parameters. In turn, the parameters form 
pairs with a key and a value. 

The q parameter is often used for search 
queries. Such queries are the most complex 
because they require checking the conditions 
of a full search, partial, and in some cases 
autocomplete. 

The formation of a uniform data set 
complicates the testing process due to a large 
number of duplicates. That is why the 
approach proposed in this paper allows you to 
replace duplicates with randomly generated 
characters that are used as a test data set. In 
this case, testing will be as close as possible 
to user input. 

It is also worth mentioning _ that 
duplication of tests is reduced by selecting a 
test data set. 


Objective 

The main purpose of this work is to use 
the automatic generation of realistic data 
taking into account the synthetic method for 
the automated testing process. 

In most cases, a separate framework is 
developed to ensure uniformity and realism of 
type testing as a client. In addition, access to 
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the web service is external, not directly 
through the code. In such cases, the web 
service is considered as a black box with 
known inputs and expected outputs. The 
inputs mean that there are related to business 
requirements and documentation. This allows 
you to develop test cases based on various 
techniques, such as specification-based and 
structured-based. This is especially important 
when the outputs are the values that in the 
tests should match the expected results. 


Testing results 

The process of automated testing 
consists of the selection of test tools for the 
framework, the formation of test scenarios, 
writing the tests themselves, and analyzing 
the results. 

In fig. 2 shows the main tools used to 
develop the framework and characterize the 
interaction between them. The central element 
of the system is TestNG, which is used to 
configure the test, as by analogy with fig.1 it 
is a formed test layer. This tool provides 
communication with other components of the 
framework. 


y4 - Ol 


nest ASSURED & -_ TestN: —»+Maven = TF) Allure 


Assert] 


Fig. 2. Interaction between the main tools in the 
developed framework for automated testing 


Equivalence partition and boundary 
value analysis can be used in the case of 
testing the web service of the basic test design 
techniques of black-box testing. 

Relevant test data are formed on the 
basis of test design techniques according to 
the following rules: 

- No data. Empty values for text fields 
or null values for numeric values 

- With correct and incorrect data 

- Illegal symbols 

- Boundary conditions [13] 

Figure 3 shows the basic rules for 
forming many types of data sets. 

The process of data generation is shown 
in Fig.4. The input components are real data 
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from the Java Faker library, a file with illegal 
symbols for generating incorrect queries, and 
a list of popular queries for compiling 
Statistics of the most used parts of the 
functionality. 


Boudary 
condition 
data No data 
“number: 0} {“number™: null} 
{"aumber™: -1}) ®\ ff 
number”: | \ / 
Test : 
IHegal data Valid data 
{"number™ + Data —> |"number 
Sets 
| 
’ 
Invalid data 
!“pumber™; 2147483648 


Fig. 3. Description of test data set options 
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4 Process 
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Fig. 4. The process of data generation 


Popular queries can be obtained using 
New Relic [12] or from other monitoring 
tools. 

The generation process is based on the 
random extraction of each data element and 
the formation of a complete record in the 
appropriate format in the file. 

Entity Generation is based on 
serialization/deserialization processes from 
objects of a certain format and vice versa. The 
main file types are Excel Binary File Format 
(XLS), Comma-Separated Values (CSV), and 
JavaScript Object Notation (JSON) for storing 
test data. 

Obtaining and visualizing the results is 
an important step in the development of 


automated tests because it allows you to 
analyze the test results. 
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Fig. 5. Graphical user interface with test results 
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Fig. 6. Detailed test results 


In fig. 5 shows the graphical interface of 
the tool for reporting Allure with test results. 
166 different test data sets have been 
generated in this run. In addition, we can see 
from the report that the path rate is 
approximately 84%. The next step is to 
analyze the list of failed tests to correct errors 
in the web service. 
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After restarting the procedure several 
times, we are constantly improving the 
software. In this case, unpredictable 
combinations may occur that the tester may 
miss when compiling the test scenario. 

As you can see in fig. 6, deploying all 
the test cases, you can determine the cause of 
failure of a particular scenario. In this case, 
the search engine did not use illegal 
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characters such as‘$ and others. In addition, 
you can analyze the search capabilities of the 
web service in terms of the full or partial 
search for information. 


Conclusion 

The paper presents one of the examples 
of the use of artificial intelligence in the 
process of writing and implementing a 
method of automated testing. Namely, 
synthetic data are used with the formation of 
ready-made data sets, as well as taking into 
account the techniques of developing test 
cases. 

Most sites generate page content based 
on responses from web services. The most 
difficult to cover and most used are the search 
functions on the web page. Search is a fairly 
flexible property on most sites. Therefore, it is 
noted that the offered technique allows to 
enter the text as full, partial, and also on the 
identifier. In partial cases, auto-replenishment 
may occur. For the correct and most similar to 
the input of data from a wide range of 
consumers need to have a real set of values, 
which are often personalized, and sometimes 
there is a limited number. 

Synthetic data are quite useful for 
applied research, as it is possible to 
implement a procedure for generating an 
artificial data set to replace real values. In 
addition, such a data set can be used for the 
process of entering data into the database and 
then the tests allow to obtain the appropriate 
realistic expectations. 

The testing process helps to reduce the 
number of errors in the software and 
constantly improve it. 

Thus, the testing method proposed in 
this paper is best suited for the data-driven 
approach and simplifies the data generation 
process. 
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